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Module 2 — ROUNDING DECIMAL NUMBERS TO A 
DESIGNATED PRECISION 

Rounding off a decimal niimber means rewriting the nximber in 
a shorter form to represent an approximation of the number. The 
symbol used for values that are approximately equal, but not 
exactly equal is In measuring the length of an object, it 
could be measured to the nearest 1 mm, the nearest 0.1 mm, the 
nearest 0.01 mm or even closer. Measurements are never exact. 
The precision with which the technician needs a measurement 
depends upon the particular application. The measurement a 
person records is a nxamber which has been read only as close as 
needed and therefore, represents a rounded off value. 

The number pi = jc = 3.141592653589793238 is a never 

ending decimal. It is necessary to round it off somewhere. 
Various possibilities includa: 



nearest 






^ 3 




nearest 


tenth 




^ 3. 


1 


neazest 


hundredth 




^ 3. 


14 


nearest 


thousandth 




^ 3. 


142 


nearest 


ten-thousandth 


. . . 7C 


^ 3. 


1416 



The proper value to select depends upon the demand for pr€.cision. 

The process called rounding off will follow the commonly 
used stops in what is called the rule--of-5. For example, when 
the number 748.537 is rounded to the units place, it is written 
as 749. Rounded to the nearest tenths, 748.537 becomes 748.5, 
and rounding to the nearest hundredth makes it 748.54. The 
following is the usual rule-of-5 for rounding a number written 
out in detail. 
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RULE FOR ROUNDING DECIMAlL NUMBERS 
ROUNDING TO TEN OR HIGHER PIACE 

To round a number to a particular place value, called the 
rounding place, that is in the ten (10) place or greater 

1. If the digit immediately to the right of the rounding 
place is less than 5 (0,1,2,3,4), then 

(a) do not change the digit in the rounding place or 
any digits to the left of it. 

(b) replace, with zeros, all the digits to the right 
of the rounding place until the decimal point 
location is reached. 

(c) do not write the decimal point. 

(d) drop all digits which were to the right of the 
decimal point (do not replace them with zeros) . 

2. If the digit iimnediately to the right of th^ rounding 
place is 5 or greater (5,6,7,8,9), then 

(a) add 1 (one) to the digit in the rounding place. 
When adding 1 to a 9 digit, add the 1 to the two 
digit number which ends in that 9. This has the 
effect of changing the 9 into 0 and the digit to 
its left will become 1 greater. 

(b) replace, with zeros, all the digits to the right 
of the rounding place until the decimal point 
location is reached. 

(c) do not write the decimal point, 

(d) drop all digits which wera to the right of the 
decimal point (ao not replace them with zero) . 

To round a number to a particular place value that is in the 
units place or to the right of the decimal point: 

3. If the digit immediately to the right of the rounding 
place is less than 5 (0,1,2,3,4), then 

(a) do not change the digit in the rounding place or 
any digits to the left of it. 

(b) drop all the digits to the right of the rounding 
place. 

(c) do not replace dropped digits with zeros. 

4. If the digit immediately to the right of the rounding 
place is 5 or greater (5,6,7,8,9), then 

(a) add 1 (one) to the digit in the rounding place. 
When adding 1 to a 9 digit, add the 1 to the two 
di jit number which ends in that 9. This has the 
effect of changing the 9 into 0 and the digit to 
its left will become 1 greater. 

(b) drop all the digits to the right of the rovnding 
place . 

(c) do not replace dropped digits with zeros. 
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EXAMPLE i: Round each number in the left-hand column to the 
place value (precision) indicated in each column 
heading . 



decimal 


ten 


unit 


tenth 


hundredth 


thousandth 


275.8103 


280 


276 


275.8 


275.81 


275.810 


43. 9618 


40 


44 


44.0 


43.96 


43.962 


27.8205 


30 


28 


27.8 


27. 82 


27.821 


7.0261 


10 


7 


7.0 


7.03 


7.026 



A special case arose when 7.0261 was rounded to the nearest 
ten. The original tens digit is considered to be 0 so that 
7.0261 = 07.0261. Since the digit immediately to the right 
of the ten digit is 7 (5 or more), then the tens digit of 0 
is to be increased by 1 to form 10. Rounding 7.0261 to the 
nearest ten produced lU. 

PRACTICE PROBLEMS: Round each number in the left-hand column to 

the place value indicated in each column 
heading . 

Decimal hundred ten unit tenth 



1. 685.31 

2. 1728.47 

3. 475.296 

4. 88.972 



Round each number in the left-hand column to the 
place value indicated in each column heading. 

Deciraal tenth hundredth thousandth ten-thousandth 



5. 1.70062 

6. 0.39491 

7. 3.06177 

8. 6.07939 

9. 0.17316 
10. 0.05592 
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A ruler, caliper and micrometer are three different devices 
used to make a length measurement. The biggest difference in 
these three instruments is the precision with which you are able 
to determine size. A person trained to use these might record 
the following for the measurement of the length of a pin. 



The PRECISION of each measurement is the place value of the 
last digit recorded by the measuring instrument and the unit of 
measure used by the instrument. 

The last digit of the metric caliper reading is the 3 of 
17.3 mm. This 3 is in the tenths or 0.1 place value position. 
The precision of the 17.3 mm caliper reading is stated as 0.1 mm. 

An English caliper reading for length of the same pin is 
0,68 in. The last digit 8 of 0.68 in. is in the hundredths or 
0,01 place value position. The precision of the 0.68 in. English 
caliper reading is stated as 0.01 in. 

EXAMPLE 2: The precision of the measurements given for the pin 
length by the various types of instruments is 
summarized in the table. 



Type of Metric Metric English English 

Device Measure Precision Measure Precision 



ruler 17mm 1mm 0.7 in. 0.1 in. 

caliper 17.3 mm 0.1 mm 0.68 in. 0.01 in. 

micrometer 17.274 mm 0.001 mm 0.6801 in. 0.0001 in. 



As one learns production skills of machining, it is 
necessary to match the quality demanded of the craftsmanship to 
the precision of the measurements required to meet those quality 
standards . 



ruler, metric 17 mm 
caliper, metric 17.5 mm 
micrometer, metric 17.274 mm 



ruler, English 0.7 in. 
caliper, English 0.68 in. 
micrometer, English 0.6801 in. 
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PRACTICE PROBLEnS: State the precision of the following 

measurements. Do not forget to include the 
unit of measurement with your niimber value. 



11. 2.4 in. 

13. 1.931 in 

IS. 7.09 in. 

17. 91.0 mm 

19, 0.81 in. 



12. 6.08 mm 

14. 12.280 mm 

16. 91 mm 

18. 91.000 mm 

20. 0.810 in. 



The English length of pin measurements of Example 2 show 
that different instruments have different precisions. The least 
precise English measure is achieved by the English ruler as 
0.7 in., while the most precise English measure is 0.6801 in. 
obtained by the English micrometer. 

The LEAST PRECISE of two or more measurements is the 
measure whose last recorded digit is farthest left 
(higher place value) . 

EXAMPLE 3: The diameter of a pin has been recorded as 0.25 in. 

and as 0.248 in. What is the precision of each 
measurement and which measurement is the least 
precise? 

Solution: 

0.25 in. has precision 0.01 in. 
0.248 in. has precision 0.001 in. 
The least precise is 0.25 in. 



EXAMPLE 4: The thickness of a steel plate has been recorded as 
9.78 mm, 9.780 mm and as 9.8 mm. What is the 
precision of each measurement and which measurement 
is the least precise? 

Solution: 

9.78 mm has precision 0.01 mm 
9.780 mm has precision 0.001 mm 
2^.8 mm has precision 0.1 mm 
The least precise is the 9.8 mm. 
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PRACTICE PROBLEMS: Identify the least precise of each set of 

measurements : 

21. 2.181 in., 2.18 in., 2.1814 in. 

22. 7.23 mm, 7.2346 mm, 7.2 mm, 7.235 mm. 

23. 1230 mm, 1200 mm, 1232.1 mm, 1232 mm 

24. 6.26 in., 6 in., 6.3 in., 6.277 in. 

25. 4.00 in., 4.0 in., 4.000 in., 4 in. 

The widi-h of 3 separate steel plates are recorded below. 
Identify the least precise of each set of measurements. 

26. 0.3 in., 2.56 in., 1.772 in. 

27. 12.3 mm, 10.44 mm, 13 mm 

28. 6.002 mm, 10.92 mm, 15.8 mm. 

29. 14.125 in., 3.25 in., 0.875 in. 

30. 9.40 mm, 27.900 mm, 316.0 mm 



ERIC 



9 



7 



SOLUTIONS TO PRACTICE PROBLEMS — MODULE 2 





Decimal 


hundred 


ten 


unit 


tenth 


1. 


685.31 


700 


690 


685 


685.3 


2. 


1728.47 


1700 


1730 


1728 


1728.5 


3. 


475.296 


500 


480 


475 


475.3 


4. 


88.972 


100 


90 


89 


89.0 





Decimal 


tenth 


hundredth 


thousandth 


ten-thou 


5. 


1.70862 


1.7 


1.71 


1.709 


1.7086 


6. 


0.39491 


0.4 


0.39 


0.395 


0.3949 


7. 


3.06177 


3.1 


3.06 


3.062 


3.0618 


8. 


6.07939 


6.1 


6.08 


6.079 


6.0794 


9. 


0.17316 


0.2 


0.17 


0.173 


0.1732 


10. 


0.05592 


0.1 


0.06 


0.056 


0.0559 



11. 0.1 iu. 

14. 0.001 mm 

17. 0.1 mm 

20. 0.001 in 

23. 1200 mm 

26. 0.3 in. 

29. 3.25 in. 



12. 0.01 mm 

15. 0.01 in. 

18. 0.001 mm 

21 . 2.18 in. 

24. 6 in. 

27. 13 mm 

30. 316.0 mm 



13. 0.001 in 

16. 1 mm 

19. 0.01 in. 

22. 7.2 mm 

25. 4 in. 

28. 15.8 mm 
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